Abstract: De-duplication is an abbreviated term to remove repeated copies of same data. It is necessary to keep most confidential data that is most susceptible while supporting de-duplication. We proposed data de-duplication system with an improved reliability as well as data confidentiality. We proposed data de-duplication process to reduced or replaced repeated data with same available data on cloud. This mainly results into saving of memory space. Data de-duplication is widely used to backup of data as well as it minimizes network overheads. It keeps only one single physical copy of data. To protect private data we used some secret sharing techniques such as Ramp secret sharing technique. Our model firstly, generates file block and then distribute them on different server. We used SHA-1 algorithm to generate block of data and hash values. Our proposed technique is totally different from existing de-duplication system as our system provides a smart solution for encrypted file duplication along with this; it must able to data secrecy as far as file is concern. To address challenges in unauthorized access, we described some privilege access. Therefore, privilege as well as non-privilege data can manage. Our system can work efficiently to reduced repetition in file level and block level data in distributed system. In this system, as a part of contribution our system will hide user’s identity and trustee server is introduced to authorize the user. Also relative address concept for data blocks is used that keeps block data integrity and encrypted data more secure from cloud or server.

Keywords: De-duplication, distributed storage system, reliability, secret sharing, relative address.